From: Xxxxxxxx Xxxxxx <xxxxxxxx.xxxxxx@ieee.org> Subject: free subscription to Debian Developer 8217 A205 5E57 043B 2883 054E 7F55 BB12 A40F 862EThis is not a signature, it s a fingerprint. Amusingly, it s not the fingerprint for the person who sent my mail, but that of Neil McGovern a co-worker at Collabora. Neil assured me he knew how to sign a mail properly, so I shitcanned that entry.
From: "Xxxxx, Xxxxxxxxx" <x.xxxxx@bbw-bremen.de> Subject: Incoming! Hey dude, I want to have the redemption code you are offering for the Valve Games mQGiBEVhrscRBAD4M5+qxhZUD67PIz0JeoJ0vB0hsLE6QPV144PLjLZOzHbl4H3N ...snip... Lz8An1TEmmq7fltTpQ+Y1oWhnE8WhVeQAKCzh3MBoNd4AIGHcVDzv0N0k+bKZQ=3D=3D =3Du/4RWat? Learn to GPG!
From: Xxxxxx-Xxxx Le Xxxxxxx Xxxx <xx.xxxxxxxxx@gmail.com> Subject: pass steam Hey me voila Merci beaucoup valve 2069 1DFC C2C9 8C47 9529 84EE 0001 8C22 381A 7594Like the first one, a fingerprint. This one is for S bastien Villemot. Don t scammers know how to GPG sign?
From: "Xxxxxxxxx Xxxxxxx" <xxxxxxxx@web.de> Subject: thanks /DD/Steam gifts us finally something back 0x6864730DF095E5E4Yet again, a fingerprint. This one is for Marco Nenciarini. I found this request particularly offensive due to the subject line the haughty tone from an identity thief struck me as astonishingly impertinent. Still, when will scammers learn to GPG?
From: Sven Hoexter <svenhoexter@gmail.com> Subject: Valve produced games I'm would like to get the valve produced games My keyring: 0xA6DC24D9DA2493D1 Sven Hoexter <hoexter> sig:6Easily the best scam effort, since this is the only one which both a) registered an email address under the name of a DD, and b) used a fingerprint which actually corresponds to that human. Sadly for the scammer, I m a suspicious kind of person, so my instinct was to verify the claim via IRC.
31-01-2014 16:52:48 > directhex: Hoaxter, have you started using gmail without updating your GPG key? (note: significantly more likely is someone trying to steal your identity a little to steal valve keys from collabora) 31-01-2014 16:54:51 < Hoaxter!~sh@duckpond6.stormbind.net: directhex: I do not use any Google services and did not change my keySo yeah. Nice try, scammer. I m not listing, in all of this, the mails which Neil received from people who didn t actually read his mail to d-d-a. I m also not listing a story which I ve only heard second ha actually no, this one is too good not to share. Someone went onto db.debian.org, did a search for every DD in France, and emailed every Jabber JID (since they look like email addresses) asking them to forward unwanted keys. All in all, the number of evildoers is quite low, relative to the number of legitimate claims 12 baddies to 279 legitimate keys issued. But still, this is why the whole key issuing thing has been taking me so long and why I have the convoluted signature-based validation system in place. Enjoy your keys, all 279 of you (or more by the time some of you read this). The offer has no explicit expiry on it Valve will keep issuing keys as long as there is reason to, and Collabora will continue to administer their allocation as long as they remain our clients. It s a joint gift to the community thousands of dollars worth of games from Valve, and a significant amount of my time to administer them from Collabora.
09-07-2013 15:10:11 < alexrp!~zor@baldr.rfw.name: TIL that mul and mult are not the same thing on mips
Why is this notable? Well, MIPS processors lack a whole bunch of instructions which are commonly used in assembler. MUL is one of them it s valid MIPS assembler, which is expanded to MULT/MFLO when compiling. Call it a macro, or a mnemonic, or shorthand the preferred term is pseudoinstructions . So what s the issue?
09-07-2013 15:18:27 > directhex: mono isn't trying to use a mul instruction, right? i mean, that instruction doesn't exist as far as the cpu is concerned, it's a macro the compiler does things with
See where this is headed?
09-07-2013 15:23:59 > directhex: mini/mini-mips.c:#define USE_MUL 1 /* use mul instead of mult/mflo for multiply */
Argh.
See, the thing about MIPS pseudoinstructions is they may be real instructions on a given CPU implementation. Strictly speaking MUL isn t a standard instruction, but a given CPU might have it anyway, to make multiplication a little faster (by using only one instruction, not two, for multiplication). In this case, the Debian MIPS infrastructure is based around ICT Loongson-2E processors which don t have that extension but the upstream Mono developers were building and testing on an extended CPU, never seeing the issue themselves.
Flipping that define to 0 (and amending the instruction length setting in another file) fixed the build. Mono was running on MIPS for me for the first time ever.
Digging through the history in git showed just how annoying this implementation quirk was. USE_MUL was added in late 2008 replacing a previously used #if 1 . The mult/mflo version of the code existed in the Mono source since the first time the full MIPS port was committed in 2006, but we never saw it.
The breakage
So, with that patched to work, I added mipsel to the Experimental build of Mono which still failed. The runtime would build fine, but the class library build would fail at random times, with random meaningless stack traces. Unrepeatable. Some kind of race condition. The build would eventually succeed if I hammered make a few times, but that s no good for the Debian build daemons. Back to square one
except I had an epiphany yesterday. I have heard more than once that Loongson processors are missing a few instructions. What if one of those was being hit, intermittently? I started doing a search in places that might need to work around that kind of issue, and found this. A patch to binutils in 2009, replacing one no-op instruction with another, when /usr/bin/as is fed the -mfix-loongson2f-nop flag.
Turns out NOP is another pseudoinstruction on MIPS. Well, more of an alias. The opcode 0 000000 is Shift Left Logical with 0 registers and 0 data, which is a no-op. But on all but the latest generation of Loongson-2F chips, that opcode can, under heavy load, fail causing inconsistent state in the CPU registers. The flag to as replaces sll 0,0,0 with or $at,$at,0 , which is also a no-op instruction, but doesn t trigger the failure on Loongson-2F chips (and 2E chips, although that s not stated in the documentation).
As long as ALL your programs get fed through as , you don t have a problem, since it uses the replacement opcode but what if you use a JITter to generate your own opcodes? Oh fuck, it couldn t be
diff --git a/mono/arch/mips/mips-codegen.h b/mono/arch/mips/mips-codegen.h
index dc4df7d..1dbd1c6 100644
--- a/mono/arch/mips/mips-codegen.h
+++ b/mono/arch/mips/mips-codegen.h
@@ -334,7 +334,7 @@ enum
/* misc and coprocessor ops */
#define mips_move(c,dest,src) mips_addu(c,dest,src,mips_zero)
#define mips_dmove(c,dest,src) mips_daddu(c,dest,src,mips_zero)
-#define mips_nop(c) mips_sll(c,0,0,0)
+#define mips_nop(c) mips_or(c,mips_at,mips_at,0)
#define mips_break(c,code) mips_emit32(c, ((code)<<6) 13)
#define mips_mfhi(c,dest) mips_format_r(c,0,0,0,dest,0,16)
#define mips_mflo(c,dest) mips_format_r(c,0,0,0,dest,0,18)
Oh yes it could! Mono was using sll 0,0,0 (the recommended no-op instruction from the MIPS instruction reference manual), causing failures in my tests, because Debian s build and test infrastructure just happens to use defective silicon. And, again, upstream were unable to reproduce a problem because they use better silicon than we do.
So what now?
Well, last night I uploaded mono_3.2.3+dfsg-3, which includes the above patch to force the replacement no-op instruction. It test built fine on the porterbox, and it should (when the damn experimental buildd gets around to it), just work.
Finally.
After just under a decade, Mono packages will be available on MIPS in Debian.
And after all this time, all we had to change was 4 lines to work around 7 year old Chinese knock-off processors.
The edit
So, things are finally built.
It turns out that despite everything, the replacement NOP opcode is not enough.
If you re-read the post to the binutils list, pay close attention to:
In theory this is still not enough to fully eliminate possible hangs, but the possiblity is extremely low now and hard to be hit in real code.
It s a filthy lie. It s easy to hit the issue in real code: just do a from-source build of the whole Mono class library. With the replacement instruction it builds .NET 2.0, 3.0, 3.5, 4.0, and most of 4.5, before dying in the same way as before an improvement on failing early in the 2.0 build, but not enough.
Thankfully, 2 out of the 5 Debian mipsel build servers are not Loongson 2 they re 11 year old Broadcom SWARM developer boards. Not fast but also not broken. Luck smiled on me, and caused my build to go to one of these Broadcom machines. As a result
(experimental_mipsel-dchroot)directhex@eder:~$ mono --version head -1
Mono JIT compiler version 3.2.3 (Debian 3.2.3+dfsg-3)
It s been a long time coming.
The answer to that is, well, you sorta don t. Metro apps can exist in three states fullscreen, almost fullscreen, or vertical stripe. You re allowed to have two apps at most at the same time one mostly full screen, and one vertical stripe. So what happens if you try to *use* that? Let s take a fairly common thing I do watch a video and play Minesweeper. In this example, the video player is the current replacement for Windows Media Player, and ships by default. The Minesweeper game isn t installed by default, but is the only Minesweeper game in the Windows 8 app store which is gratis and by Microsoft Game Studios. Here s option A:
And for contrast, here s option B:
Which of these does a better job of letting me play Minesweeper and watch a video at the same time? Oh, here s option C, dumping Microsoft s own software, and using a third-party video player and third party Minesweeper implementation:
It s magical almost as if picking my own window sizes makes the experience better. So, as you can see above, the old OS is still hiding there, in the form of a Windows 8 app called Desktop . Oh, sorry, didn t I say? Metro apps, and non-Metro apps, are segregated. You can run both (the Desktop app can also be almost-fullscreen or a vertical strip), but they get their own lists of apps when multitasking. Compare the list on the left with the list at the bottom:
And it s even more fun for apps like Internet Explorer, which can be started in both modes (and you often need both modes). Oh, and notice how the Ribbon interface from Office 2007 has invaded Explorer, filling the view with large buttons to do things you never want to do under normal circumstances. So, that s a short primer on why Windows 8 is terrible. Is there really nothing here worth stealing? Actually, yes, there is! After much research, I have discovered Windows 8 s shining jewel:
The new Task Manager is lovely. I want it on my Linux systems. But that s it.
A program is free software if the program s users have the four essential freedoms:The Debian Free Software Guidelines and Open Source Initiative s Open Source Definition clause 6 reads:
- The freedom to run the program, for any purpose (freedom 0).
No Discrimination Against Fields of Endeavor The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.It s fairly clear that the JSON.org license clause goes against both of these, making any piece of software using that license neither Open Source nor Free Software. It is not distributable by any organization which mandates Freedom for its users not in Debian, not in Fedora, not on Google Code. Anybody who cares about their users will reject a clause like this, because it has awful chilling effects. Think it s funny? It really isn t. Who can use the code without being at risk of a lawyer knocking on their doors? Can the Catholic Church? The ACLU? Republicans? Democrats? Military contractors? Genetic engineers? Big pharma? Petrochemicals firms? Communists? Fascists? Mono developers? WikiLeaks? Without a clear definition of good and evil , people need to seriously consider whether they are safe to use the code because if the developer and users interpretations of the terms differ, there could be hell to pay. Would you want to ship a hardware device let s say you re making a smart TV and have some developer of a library send his lawyers around to tell you sorry, TV rots kids brains, it s clearly evil to get your entire distribution channel shut down by injunction? It s not an oversight, kids. Upstream are well aware of the pain this childishness causes anyone who takes software licenses seriously. They giggle about it at conferences, like tweenagers who think they ve one-upped the adults in the room. This non-Free license is intended to mock people who take licensing seriously. In fact, one could easily categorise efforts to pollute the Free Software ecosystem with fake, non-Free software as evil, which would mean all JSON.org code fails to comply with its own license, due to shipping with its license (Inception, anyone?). Before the jokers in the room claim that this kind of problem is deserved by Mono, let s take a look at all the software in Debian which Douglas Crockford endangers with his childishness. PHP? OwnCloud? jQuery? You think Debian serves its users well by pulling jQuery from the next release in order to serve the ego of a man behaving like an eleven year old? It is also interesting to note that the author of this do no evil clause works at PayPal. Read that twice, go and repair your irony meters, then come back. In conclusion: thank you so VERY much Doug Crockford for making the world of Free Software measurably worse.
When I was first approached about this, I thought Collabora was a small company. But as I looked more into it, I discovered that's not longer the case, there's many more people than I imagined working there (here!), and was delighted to see I knew many of them, and many other are well known members of the major Free Sofware communities. I'll be joining the sysadmin team to work closely with Jo Shields. See you tomorrow, folks. :) This opportunity to work from home is godsend, given the third bit of change that'll be happening soon: sometime in late September, Maria and I should join the ranks of first-time parents, following the baby boom wave surrounding us. While you can imagine we're really happy about this, we're also freaking out because this is going to happen in just two months and a half, and weeks go by really fast lately. So yeah, being able to be at home with this really small baby will be a big bonus for the incredible experience we're about to enjoy. We've been both busy with other stuff, but during the summer we should be focusing on preparing the baby's arrival. There's a whole lot to do! Expect my Debian and other Free Software activities to get a hit, of course. :) If I am normally sleep-deprived, this is going to be the next level.
Package: leafnode Version: 2.0.0.alpha20090406a-1 ============================= ProblemType: Crash Architecture: amd64 Date: Tue Jul 3 00:08:02 2012 Dependencies: adduser 3.113+nmu3 base-passwd 3.5.24 cron 3.0pl1-123 debconf 1.5.44 debianutils 4.3.1 dpkg 1.16.4.3 gcc-4.7-base 4.7.1-2 libbz2-1.0 1.0.6-3 libc-bin 2.13-33 libc6 2.13-33 libclass-isa-perl 0.36-3 libdb5.1 5.1.29-4 libfile-copy-recursive-perl 0.38-1 libgcc1 1:4.7.1-2 libgdbm3 1.8.3-11 liblzma5 5.1.1alpha+20120614-1 libpam-modules 1.1.3-7.1 libpam-modules-bin 1.1.3-7.1 libpam-runtime 1.1.3-7.1 libpam0g 1.1.3-7.1 libpcre3 1:8.30-5 libpopt0 1.16-7 libselinux1 2.1.9-5 libsemanage-common 2.1.6-6 libsemanage1 2.1.6-6 libsepol1 2.1.4-3 libswitch-perl 2.16-2 libustr-1.0-1 1.0.4-3 libwrap0 7.6.q-23 logrotate 3.8.1-4 lsb-base 4.1+Debian7 [modified: lib/lsb/init-functions] multiarch-support 2.13-33 netbase 5.0 openbsd-inetd 0.20091229-2 passwd 1:4.1.5.1-1 perl 5.14.2-12 perl-base 5.14.2-12 perl-modules 5.14.2-12 sensible-utils 0.0.7 tar 1.26-4 tcpd 7.6.q-23 update-inetd 4.43 zlib1g 1:1.2.7.dfsg-13 Disassembly: => 0x7f8e69738475 <*__GI_raise+53>: cmp $0xfffffffffffff000,%rax 0x7f8e6973847b <*__GI_raise+59>: ja 0x7f8e69738492 <*__GI_raise+82> 0x7f8e6973847d <*__GI_raise+61>: repz retq 0x7f8e6973847f <*__GI_raise+63>: nop 0x7f8e69738480 <*__GI_raise+64>: test %eax,%eax 0x7f8e69738482 <*__GI_raise+66>: jg 0x7f8e69738465 <*__GI_raise+37> 0x7f8e69738484 <*__GI_raise+68>: test $0x7fffffff,%eax 0x7f8e69738489 <*__GI_raise+73>: jne 0x7f8e697384a2 <*__GI_raise+98> 0x7f8e6973848b <*__GI_raise+75>: mov %esi,%eax 0x7f8e6973848d <*__GI_raise+77>: nopl (%rax) 0x7f8e69738490 <*__GI_raise+80>: jmp 0x7f8e69738465 <*__GI_raise+37> 0x7f8e69738492 <*__GI_raise+82>: mov 0x34e97f(%rip),%rdx # 0x7f8e69a86e18 0x7f8e69738499 <*__GI_raise+89>: neg %eax 0x7f8e6973849b <*__GI_raise+91>: mov %eax,%fs:(%rdx) 0x7f8e6973849e <*__GI_raise+94>: or $0xffffffff,%eax 0x7f8e697384a1 <*__GI_raise+97>: retq DistroRelease: Debian 7.0 ExecutablePath: /usr/sbin/fetchnews ExecutableTimestamp: 1265584779 Package: leafnode 2.0.0.alpha20090406a-1 PackageArchitecture: amd64 ProcCmdline: /usr/sbin/fetchnews ProcCwd: / ProcEnviron: LANGUAGE=en_US:en LC_TIME=en_IN.UTF-8 LC_MONETARY=en_IN.UTF-8 PATH=(custom, no user) LC_ADDRESS=en_IN.UTF-8 LANG=en_US.UTF-8 LC_TELEPHONE=en_IN.UTF-8 LC_NAME=en_IN.UTF-8 SHELL=/bin/sh LC_MEASUREMENT=en_IN.UTF-8 LC_NUMERIC=en_IN.UTF-8 LC_PAPER=en_IN.UTF-8 ProcMaps: 00400000-00421000 r-xp 00000000 08:06 4464934 /usr/sbin/fetchnews 00621000-00622000 rw-p 00021000 08:06 4464934 /usr/sbin/fetchnews 00622000-00623000 rw-p 00000000 00:00 0 00be4000-00c05000 rw-p 00000000 00:00 0 [heap] 7f8e68edc000-7f8e68eef000 r-xp 00000000 08:06 1179776 /lib/x86_64-linux-gnu/libresolv-2.13.so 7f8e68eef000-7f8e690ee000 ---p 00013000 08:06 1179776 /lib/x86_64-linux-gnu/libresolv-2.13.so 7f8e690ee000-7f8e690ef000 r--p 00012000 08:06 1179776 /lib/x86_64-linux-gnu/libresolv-2.13.so 7f8e690ef000-7f8e690f0000 rw-p 00013000 08:06 1179776 /lib/x86_64-linux-gnu/libresolv-2.13.so 7f8e690f0000-7f8e690f2000 rw-p 00000000 00:00 0 7f8e690f2000-7f8e690f7000 r-xp 00000000 08:06 1180036 /lib/x86_64-linux-gnu/libnss_dns-2.13.so 7f8e690f7000-7f8e692f6000 ---p 00005000 08:06 1180036 /lib/x86_64-linux-gnu/libnss_dns-2.13.so 7f8e692f6000-7f8e692f7000 r--p 00004000 08:06 1180036 /lib/x86_64-linux-gnu/libnss_dns-2.13.so 7f8e692f7000-7f8e692f8000 rw-p 00005000 08:06 1180036 /lib/x86_64-linux-gnu/libnss_dns-2.13.so 7f8e692f8000-7f8e692fa000 r-xp 00000000 08:06 1183392 /lib/libnss_mdns4_minimal.so.2 7f8e692fa000-7f8e694f9000 ---p 00002000 08:06 1183392 /lib/libnss_mdns4_minimal.so.2 7f8e694f9000-7f8e694fa000 rw-p 00001000 08:06 1183392 /lib/libnss_mdns4_minimal.so.2 7f8e694fa000-7f8e69505000 r-xp 00000000 08:06 1180055 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7f8e69505000-7f8e69704000 ---p 0000b000 08:06 1180055 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7f8e69704000-7f8e69705000 r--p 0000a000 08:06 1180055 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7f8e69705000-7f8e69706000 rw-p 0000b000 08:06 1180055 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7f8e69706000-7f8e69883000 r-xp 00000000 08:06 1179700 /lib/x86_64-linux-gnu/libc-2.13.so 7f8e69883000-7f8e69a83000 ---p 0017d000 08:06 1179700 /lib/x86_64-linux-gnu/libc-2.13.so 7f8e69a83000-7f8e69a87000 r--p 0017d000 08:06 1179700 /lib/x86_64-linux-gnu/libc-2.13.so 7f8e69a87000-7f8e69a88000 rw-p 00181000 08:06 1179700 /lib/x86_64-linux-gnu/libc-2.13.so 7f8e69a88000-7f8e69a8d000 rw-p 00000000 00:00 0 7f8e69a8d000-7f8e69a95000 r-xp 00000000 08:06 1180031 /lib/x86_64-linux-gnu/libcrypt-2.13.so 7f8e69a95000-7f8e69c94000 ---p 00008000 08:06 1180031 /lib/x86_64-linux-gnu/libcrypt-2.13.so 7f8e69c94000-7f8e69c95000 r--p 00007000 08:06 1180031 /lib/x86_64-linux-gnu/libcrypt-2.13.so 7f8e69c95000-7f8e69c96000 rw-p 00008000 08:06 1180031 /lib/x86_64-linux-gnu/libcrypt-2.13.so 7f8e69c96000-7f8e69cc4000 rw-p 00000000 00:00 0 7f8e69cc4000-7f8e69cc6000 r-xp 00000000 08:06 1180047 /lib/x86_64-linux-gnu/libdl-2.13.so 7f8e69cc6000-7f8e69ec6000 ---p 00002000 08:06 1180047 /lib/x86_64-linux-gnu/libdl-2.13.so 7f8e69ec6000-7f8e69ec7000 r--p 00002000 08:06 1180047 /lib/x86_64-linux-gnu/libdl-2.13.so 7f8e69ec7000-7f8e69ec8000 rw-p 00003000 08:06 1180047 /lib/x86_64-linux-gnu/libdl-2.13.so 7f8e69ec8000-7f8e69ed5000 r-xp 00000000 08:06 1179893 /lib/x86_64-linux-gnu/libpam.so.0.83.0 7f8e69ed5000-7f8e6a0d4000 ---p 0000d000 08:06 1179893 /lib/x86_64-linux-gnu/libpam.so.0.83.0 7f8e6a0d4000-7f8e6a0d5000 r--p 0000c000 08:06 1179893 /lib/x86_64-linux-gnu/libpam.so.0.83.0 7f8e6a0d5000-7f8e6a0d6000 rw-p 0000d000 08:06 1179893 /lib/x86_64-linux-gnu/libpam.so.0.83.0 7f8e6a0d6000-7f8e6a112000 r-xp 00000000 08:06 1182916 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f8e6a112000-7f8e6a312000 ---p 0003c000 08:06 1182916 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f8e6a312000-7f8e6a313000 rw-p 0003c000 08:06 1182916 /lib/x86_64-linux-gnu/libpcre.so.3.13.1 7f8e6a313000-7f8e6a333000 r-xp 00000000 08:06 1180134 /lib/x86_64-linux-gnu/ld-2.13.so 7f8e6a503000-7f8e6a507000 rw-p 00000000 00:00 0 7f8e6a530000-7f8e6a532000 rw-p 00000000 00:00 0 7f8e6a532000-7f8e6a533000 r--p 0001f000 08:06 1180134 /lib/x86_64-linux-gnu/ld-2.13.so 7f8e6a533000-7f8e6a534000 rw-p 00020000 08:06 1180134 /lib/x86_64-linux-gnu/ld-2.13.so 7f8e6a534000-7f8e6a535000 rw-p 00000000 00:00 0 7fffbc06e000-7fffbc08f000 rw-p 00000000 00:00 0 [stack] 7fffbc1ff000-7fffbc200000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] ProcStatus: Name: fetchnews State: S (sleeping) Tgid: 6872 Pid: 6872 PPid: 6871 TracerPid: 0 Uid: 9 9 9 9 Gid: 9 9 9 9 FDSize: 64 Groups: 9 VmPeak: 21440 kB VmSize: 21276 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 984 kB VmRSS: 984 kB VmData: 380 kB VmStk: 136 kB VmExe: 132 kB VmLib: 2132 kB VmPTE: 64 kB VmSwap: 0 kB Threads: 1 SigQ: 0/23227 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000000000 SigCgt: 0000000000000000 CapInh: 0000000000000000 CapPrm: 0000000000000000 CapEff: 0000000000000000 CapBnd: ffffffffffffffff Cpus_allowed: f Cpus_allowed_list: 0-3 Mems_allowed: 00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 6 nonvoluntary_ctxt_switches: 1 Registers: rax 0x0 0 rbx 0x0 0 rcx 0xffffffffffffffff -1 rdx 0x6 6 rsi 0x1ad8 6872 rdi 0x1ad8 6872 rbp 0x0 0x0 rsp 0x7fffbc08d4f8 0x7fffbc08d4f8 r8 0x7f8e6a504700 140249645729536 r9 0x6d6f642064656966 7885631562835126630 r10 0x8 8 r11 0x246 582 r12 0x0 0 r13 0x7fffbc08d920 140736348084512 r14 0x0 0 r15 0x0 0 rip 0x7f8e69738475 0x7f8e69738475 <*__GI_raise+53> eflags 0x246 [ PF ZF IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 Signal: 6 SourcePackage: leafnode Stacktrace: #0 0x00007f8e69738475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 pid = <optimized out> selftid = <optimized out> #1 0x00007f8e6973b6f0 in *__GI_abort () at abort.c:92 act = __sigaction_handler = sa_handler = 0x7f8e69850d3d, sa_sigaction = 0x7f8e69850d3d , sa_mask = __val = 140736348083740, 140249634747968, 0, 0, 140736348084512, 140249631083424, 140249645738440, 0, 4294967295, 206158430232, 1, 6427272, 0, 0, 0, 0 , sa_flags = 1781664242, sa_restorer = 0x1 sigs = __val = 32, 0 <repeats 15 times> #2 0x0000000000416292 in ?? () No symbol table info available. #3 0x0000000000411c80 in ?? () No symbol table info available. #4 0x0000000000406952 in ?? () No symbol table info available. #5 0x00007f8e69724ead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228 result = <optimized out> unwind_buf = cancel_jmp_buf = jmp_buf = 0, 498162418289285118, 4206480, 140736348084512, 0, 0, -498021567843461122, -435434157313288194 , mask_was_saved = 0 , priv = pad = 0x0, 0x0, 0x418360, 0x7fffbc08d928 , data = prev = 0x0, cleanup = 0x0, canceltype = 4293472 not_first_call = <optimized out> #6 0x0000000000402fb9 in ?? () No symbol table info available. #7 0x00007fffbc08d918 in ?? () No symbol table info available. #8 0x000000000000001c in ?? () No symbol table info available. #9 0x0000000000000001 in ?? () No symbol table info available. #10 0x00007fffbc08ee96 in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available. StacktraceTop: ?? () ?? () ?? () __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228 ?? () ThreadStacktrace: . Thread 1 (LWP 6872): #0 0x00007f8e69738475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 pid = <optimized out> selftid = <optimized out> #1 0x00007f8e6973b6f0 in *__GI_abort () at abort.c:92 act = __sigaction_handler = sa_handler = 0x7f8e69850d3d, sa_sigaction = 0x7f8e69850d3d , sa_mask = __val = 140736348083740, 140249634747968, 0, 0, 140736348084512, 140249631083424, 140249645738440, 0, 4294967295, 206158430232, 1, 6427272, 0, 0, 0, 0 , sa_flags = 1781664242, sa_restorer = 0x1 sigs = __val = 32, 0 <repeats 15 times> #2 0x0000000000416292 in ?? () No symbol table info available. #3 0x0000000000411c80 in ?? () No symbol table info available. #4 0x0000000000406952 in ?? () No symbol table info available. #5 0x00007f8e69724ead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228 result = <optimized out> unwind_buf = cancel_jmp_buf = jmp_buf = 0, 498162418289285118, 4206480, 140736348084512, 0, 0, -498021567843461122, -435434157313288194 , mask_was_saved = 0 , priv = pad = 0x0, 0x0, 0x418360, 0x7fffbc08d928 , data = prev = 0x0, cleanup = 0x0, canceltype = 4293472 not_first_call = <optimized out> #6 0x0000000000402fb9 in ?? () No symbol table info available. #7 0x00007fffbc08d918 in ?? () No symbol table info available. #8 0x000000000000001c in ?? () No symbol table info available. #9 0x0000000000000001 in ?? () No symbol table info available. #10 0x00007fffbc08ee96 in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available. Title: fetchnews crashed with SIGABRT in __libc_start_main() Uname: Linux 3.4-trunk-amd64 x86_64 UserGroups:The first 2 lines should be enough for the BTS server to file the bug report to the correct package and against the correct version.If you paid attention in the screen shot and the email report, you will notice that the email report does not include the CoreDump section. This was knocked off intentionally as we currently might not need it. If there is a need, the data is always available on the user's apport database, and the dev/user and seek it on a need basis. But in case, if a day comes when we really want it, that too is possible. The same email report will have the CoreDump seciton with a section like the following:
CoreDump: base64
H4sICAAAAAAC/0NvcmVEdW1wAOx9C0AU1f7/7LLAggirouF7VDQ0xcVXWGqLoq4Kur6KMmV5CoqwwqJYVquggYqhlVm3B3mt7OWlbnXtZeubHtfobc9Lb6xUemiU5f7PmfkcmBl3VMzbvff3Px89fPb7nfOeM2fOOfOdMzeNT5pgNBgEBpMwRmiRBMEmnA6bEC8MbfYvI6eN4BePBfvX0zQC6Q9RJyULY6NKpuGC6A8rO56uE07wH44lI7YynINlc6cmnFE3n1L5nNB7j+Ssv8lPelbDaeEkuBCu8ag6HENjz9PCmZTh6sNz/aanVz4PS6+V4aoQTohQhxM1rK2XGoQTI/yn5xD814sX4VyacGerFxbOObh15atj6Z17OKl89Qjn0Qkn6pSvEeGqB59z+YKU4aqmtS6fQgDS0wlXo5NPC8I5HK07DyyczdW68yCy9FytK58V4Zw64eou8l8+G8JZy/2Xj51Abfmaw21Th7Np+LTrD+Fc21p5/SGcRxPOoX8dye0T4ep00vPotU92Hh5u3Xln4Wwvt6587A7jaGU4C8I5X/afz3qd/pqFs73WuutWZOm91rr20hzuu9aVz4pwru9adz3YEM6jE65Op3wOdh6OtO68s3C2tgtaVT4nS+/cw0nlc7H61AnnDfBfPg/Cie0WnOv5C1KFi2tdPqsQzqoTTtAZ91SzcI4Frepf6tlIbeaCc71PyyECcXxm686fBeGsrQwnIpytleGsCFcX9IJPVT6jmrXnwRHM2q863NnOn5MNbJ0+irOFo2EcBjn8uGkzxgvN/ZuMID8t7lAXQXiPuHfhLjTE983qMmswAVVmhpydm5UpZhYsEiaq254PCFLkW8nfQF9cViT1HUM18Te9LefjEhahJv6zgV3DtI7LMaJhdSzMwMFo+cIIgdPWbXaWOyMnP2tpEbwPLi4qHFyUnps/uPmI+EfqOhBD/SBF3nq19J1S+VmT8Z389iVVm8CBdhAzwQHNl 7xNNReLV6SpPH/NUzH0OWZNHkPAVYPl+Npo9GEaua1GDtfIF2ni7wz+5QP5fOOOIXxxTJYjWPgP/bfLQHSLRkUd3hTgv749PWm7FP7r0FcDP15U/cmO9fL4jLFNgzkaiBr4zwWL//QaYtey3vH/NtBrxEbbBHFJk6bOTuFt4v9cmwj4N/vn4ODg4ODg4ODg4OD4/wE3aZ7/G/H8n60B2aDfFmVs9kOf/5vJ325CV2n+HSgol59tKvYiasZmxRzNJCeIhG0q7gY1Y4OCA1UlsKl4b5hRxWxhu2Vdmq0Dp6vYi8UxS5SgCmdEuBiEi4F
Now, Is this going to be useful? Will we get flooded with bug reports?
Usefulness: I think this will uncover many bugs that might be getting unnoticed today. Take this example itself. This bug was detect against the leafnode package's fetchnews binary. That binary does a quiet job in the background, fetching news. I never was aware that it had been crashing. As a system-wide monitoring tool, apport was able to detect, trap and inform the user of the crash. So I think this will be useful and improve the overall quality.
Abuse/Flood: This is something I can't predict. Perhaps it will bring in challenges if people just blindly click on the Send Report button. One option can be to have the "Send Report" checkbox unchecked by default. That'll hopefully lower down the possibilities.
What do you think? Let me know your comments.
This change will soon be pushed to apport in experimental. Hopefully this weekend.
I want to wrap up this post with a Thank You note to Martin Pitt and his team who created Apport, and with such simplicity. It took me 30 lines of code to adapt it to Debian. My intent with Apport is to keep the changes to the minimum so that we can always leverage new features and fixes that Apport brings in. So, as far as I'll try, there will be no drift from its original shape.
puts 'Hello world'and the Mono result is the minimum install required to run the following code, which has been compiled on a different machine using dmcs :
public class Hello1 public static void Main() System.Console.WriteLine("Hello, World!");Obviously, for dynamic languages like Ruby and Python, there is no compiler step needed. For languages like C# and Java, it is fair to compile these on a different machine as end-users of packages running these frameworks receive binaries, not source they do not need a development environment. So, on to the test environment. This is a bare bootstrapped AMD64 Debian Unstable system, as of today (i.e. debootstrap sid /tmp/badger ). Its install size is 275MB. So how much space? Ruby 1.9 Test code is as follows:
puts 'Hello world'Install requirements are as follows:
root@dream:/# aptitude install ruby1.9.1 The following NEW packages will be installed: libffi5 a libruby1.9.1 a libyaml-0-2 a ruby1.9.1 0 packages upgraded, 4 newly installed, 0 to remove and 0 not upgraded. Need to get 4703 kB of archives. After unpacking 13.1 MB will be used.OpenJDK 7 Test code is as follows:
class HelloWorldApp public static void main(String[] args) System.out.println("Hello world");Install requirements are as follows:
root@dream:/# aptitude install openjdk-7-jre-headless The following NEW packages will be installed: ca-certificates a ca-certificates-java a dbus a fontconfig-config a icedtea-7-jre-cacao a icedtea-7-jre-jamvm a java-common a krb5-locales a libavahi-client3 a libavahi-common-data a libavahi-common3 a libcap2 a libcups2 a libdbus-1-3 a libexpat1 a libffi5 a libfontconfig1 a libfreetype6 a libglib2.0-0 a libglib2.0-data a libgssapi-krb5-2 a libjpeg8 a libk5crypto3 a libkeyutils1 a libkrb5-3 a libkrb5support0 a liblcms2-2 a libnspr4 a libnss3 libnss3-1d a libpcre3 a libpcsclite1 a libsystemd-login0 a libxml2 a openjdk-7-jre-headless openjdk-7-jre-lib a openssl a sgml-base a shared-mime-info a ttf-dejavu-core a tzdata-java a ucf a xml-core a 0 packages upgraded, 43 newly installed, 0 to remove and 0 not upgraded. Need to get 51.5 MB of archives. After unpacking 137 MB will be used.Python 3.2 Test code is as follows:
print ("Hello world")Install requirements are as follows:
root@dream:/# aptitude install python3.2-minimal The following NEW packages will be installed: file a libexpat1 a libffi5 a libmagic1 a mime-support a python3.2 a python3.2-minimal 0 packages upgraded, 7 newly installed, 0 to remove and 0 not upgraded. Need to get 4853 kB of archives. After unpacking 17.7 MB will be used.Mono 2.10 (C#) Test code is as follows:
public class Hello public static void Main() System.Console.WriteLine("Hello world");Install requirements are as follows:
root@dream:/# aptitude install mono-runtime The following NEW packages will be installed: binfmt-support a cli-common a libmono-corlib4.0-cil a libmono-i18n-west4.0-cil a libmono-i18n4.0-cil a libmono-security4.0-cil a libmono-system-configuration4.0-cil a libmono-system-security4.0-cil a libmono-system-xml4.0-cil a libmono-system4.0-cil a mono-4.0-gac a mono-gac a mono-runtime 0 packages upgraded, 13 newly installed, 0 to remove and 0 not upgraded. Need to get 4383 kB of archives. After unpacking 11.5 MB will be used.Of course, every time I try to make the argument that Mono is much leaner than what people propose as an alternative, the anti-Mono brigade go on the defensive and insist that nobody in their right minds uses Python or Ruby either everything ever should be written in Qt. So, hypothetically Qt 4.8 (C++) Test code is as follows:
#include <QTextStream> int main(int argc, char **argv) QTextStream s(stdout); s << "Hello, world!\n"; return 0;Install requirements are as follows:
root@dream:/# aptitude install libqtcore4 The following NEW packages will be installed: libffi5 a libglib2.0-0 a libglib2.0-data a libpcre3 a libqtcore4 libxml2 a sgml-base a shared-mime-info a xml-core a 0 packages upgraded, 9 newly installed, 0 to remove and 0 not upgraded. Need to get 10.3 MB of archives. After unpacking 27.8 MB will be used.Presented without comment for your consideration. Edit: There are complaints that these numbers are misleading, because they show default package manager behaviour, rather than what happens when passing extra flags or config file settings to Apt. When disabling installation of Recommends: packages, the numbers change as follows:
Only ONE of us is the kind of person that goes up to guys with machine guns to ask what s happening. Me to Terah todayThey told me that it was the preparations for the opening ceremony for a global shooting contest, and also gave me directions to the bus stop.
strncpy
is a safer alternative to strcpy
. It's not. It's a slower function
that is not going to make you significantly safer.
The problem with strcpy
is that it's somewhat easy to overflow
the target array:
char src[] = "hello, world";
char dst[5];
strcpy(dst, src); // Oops.
The suggested alternative with strncpy
would be this:
strncpy(dst, src, sizeof(dst));
strncpy
gets the size of the destination array as a parameter,
and can therefore make sure the string will fit there. Problem solved?
Not so much.
Every time you do any string handling with C's native string
types and functions, you need to make sure you have allocated
enough memory for each string, you need to keep track of
how much memory you've allocated, and you neet to make sure the
zero byte ('NUL', '\0'
) is put there correctly. This is a lot
of details to keep track of, and it's not surprising things
sometimes go wrong.
If your arrays are allocated statically, which is the simplest,
but least useful case, you need to something like this with
strcpy
:
if (strlen(src) + 1 > sizeof(dst))
panic(); // or however you want to handle your errors
strcpy(dst, src);
The +1
after strlen
is to make room for the zero byte (NUL,
'\0'
) that is used in C to indicate the end of a string.
With strncpy
, your code is slightly simpler. This
is its attraction.
Problem is, this only works if dst
is an array defined within
the scope of the code. If, as is common, you have received dst
as an argument to a function, sizeof(dst)
is the size of a
pointer to an element of the array, not the size of the array.
So now you need to track the size explicitly in a second variable:
strncpy(dst, src, dst_size);
Further, strncpy
has an unfortunate error handling mode: it merely fills
the destination array and stops. It does not indicate failure in
any way. It does not terminate the target array with a zero
byte. Since all C string processing assume the zero byte is there,
as soon as you try to use dst
as a string, you're in trouble.
But luckily you can check for that before you call strncpy
:
if (strlen(src) + 1 > dst_size)
panic(); // or whatever
strncpy(dst, src, dst_size);
By this time, the code is equally hard whether you use
strcpy
or strncpy
. With strncpy
you often waste some
extra time, though, filling the unused parts of the target
array with zero bytes: they're almost never necessary.
Alternatively, you can force the zero byte there yourself,
silently truncating a string.
strncpy(dst, src, dst_size);
dst[dst_size - 1] = '\0'; // let's hope dst_size > 0
If your string handling is OK
with strings being randomly shorter than they should be,
possibly you could replace all of them with empty strings,
and save yourself a world of pain.
Instead of strncpy
, use snprintf
. It has sensible
error behavior, and is much more versatile:
n = snprintf(dst, dst_size, "%s", src);
if (n >= dst_size)
panic();
snprintf
still isn't very nice to use. It still requires you
to allocate memory yourself, and keep track of the size of each
array manually. It's also slower for small destination buffers
(but faster for large ones).
The real solution is to abandon C's antiquated string handling,
and use a more sensible helper library instead. There's a bunch
of them around, though none have gained much popularity. I'm
not going to name any of them,
but just imagine how much nicer
it would be to write code like this instead:
Dynstr *src = dynstr_new_from_cstring("hello, world\n");
Dynstr *cat = dynstr_cat_many(src, src, src, src, src, NULL);
Dynstr *dog = dynstr_substr(cat, 15, 5);
Let the library take care of memory allocations and related
tracking, and concentrate on real problem solving instead.
We did this in Kannel, back in 1999,
and it wiped out all the buffer overflow problems (until a
co-worker decided it was "more efficient" to use native C strings
instead, promptly introducing a buffer overflow).
All of this string handling is, of course, way nicer in higher
level languages. In fact, whenever you need to deal with string
handling in C, it is probably a sign that you should consider switching
to a higher level language instead.
But whatever you do, stop claiming that strncpy
is safe or
solves any real problems strcpy
has.
Update: Fixed "It does terminate" to say "It does not terminate".
Thanks to Matthew Garrett for pointing that out.
Update2: Fixed example call to snprintf
.
Update3: Fixed the test for snprintf
example.
Thanks, Redditor axylone.
Also pointed out that snprintf
is slower than the rest.
Next.